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Abstract 


Recently, studies have adopted fixed effects modeling to identify the returns to college. 
This method has the advantage over ordinary least squares estimates in that unobservable, 
individual-level characteristics that may bias the estimated returns are differenced out. But the 
method requires extensive longitudinal data and involves complex specifications, raising the 
possibility that results are sensitive either to sample restrictions or to alternative specifications. 
Also, the extra requirements might not be justified if results from fixed effects models are 
broadly similar to those from conventional ordinary least squares models. In this paper we 
review results from fixed effects models of the earnings gains from completing an associate 
degree relative to non-completion for community college students. We examine both sampling 
restrictions and specification issues. We find results to be sensitive to assumptions about missing 
earnings data and to how time trend specifications are modeled. However, we find no 
substantively meaningful differences between estimates using fixed effects models and ordinary 
least squares methods. A main benefit of fixed effects models — controlling for unobservable 
student characteristics — should be weighed against the difficulty in interpreting coefficients and 
more intensive data requirements. On the other hand, a distinct advantage of fixed effects models 
is that they allow for analysis of earning profiles over the period from before to after college. 
Given the large fluctuations in earnings over this period, this advantage may be significant in 
yielding evidence on the full returns to college. 
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1. Introduction 


For many decades, the labor market returns to college have been estimated using the 
Mincerian Ordinary Least Squares (OLS) functional form (Barrow & Malamud, 2015). Although 
robust and straightforward to interpret, the well-known limitation of this form is that it does not 
pennit causal interpretation: evidence of higher earnings for degree holders may reflect 
unobserved ability or motivation (Rouse, 2007). One response to this limitation is the use of 
fixed effects (FE) models (since Jacobson, LaLonde, & Sullivan, 2005). Fixed effects models 
estimate the gains from college as the individual-level increase in earnings between time periods 
(usually quarters of years) when individuals have an award compared with time periods when 
they do not have an award. The advantage of this approach is that within-person characteristics 
that are typically unobserved (e.g., ability) should be differenced out of the identified returns. 
This fixed effects approach has been made possible by the availability of large-scale student- 
level administrative datasets from postsecondary education systems (see Figlio, Karbownik, & 
Salvanes, 2015). On average, conclusions from these analyses correspond with the OLS 
consensus: more intensive postsecondary education has effects on earnings that are positive, 
economically meaningful, sustained over the career and business cycle, and consistent across 
states and institutions. 1 These conclusions do not hold for all college pathways but are robust for 
associate degrees (see Belfield & Bailey, 2017). ~ 

However, FE models may have some methodological drawbacks. In a review of studies 
across social sciences, Sobel (2012, p. 527) concludes that “researchers ... have not defined the 
effects they are attempting to estimate or understood the types of conditions that must be met in 
order to estimate treatment effects.... This can lead to incorrect conclusions and misguided 
policy recommendations.” We identify two potential drawbacks. One is that FE model 
specifications are significantly more complicated and involve more variables than basic OLS 
specifications. Notably, FE models involve assumptions about time trends and earnings 
trajectories of students during and after college (Henderson, Polachek, & Wang, 2011). Results 
may therefore be sensitive to model specifications. The other issue is that FE models are more 
data intensive: they require data before, during, and after college. If these data are missing or 
unavailable (e.g., for students with no work experience before they enter college), then results 
may be biased. Even as FE models should eliminate biases from individual-level unobservable 
characteristics, the sample restrictions and model specification complexities may render these 
models non-robust. Moreover, if their results are broadly equivalent to results from OLS models, 
it may not be worthwhile to collect more intensive data and run more complex FE models. 


1 By more intensive, we mean that the student either accumulates more credits or earns an award. Evidence for these 
claims is reviewed in the synthesis by Belfield and Bailey (2017). See also Barrow and Malamud (2015) and 
Oreopoulos and Petronijevic (2013). 

" Earnings gains from shorter term certificates dissipate a few years after exiting college. Gains from associate 
degrees in liberal arts or general studies are quite low; although if these students transfer to a four-year institution 
the returns are high (see Dadgar &Trimble, 2015; Liu, Belfield, & Trimble, 2015). 
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In this review paper we investigate the robustness of FE models. First, we describe the 
key differences between OLS and FE models. Next, we examine how results from FE models 
vary across different sample restrictions and model specifications. Then we examine whether 
Mincerian OLS models yield different results than FE estimations. For these analyses we use 
administrative data from North Carolina and summarize evidence from other statewide datasets. 
To clearly illustrate sampling and functional form sensitivities we focus on the returns to 
completion relative to non-completion of associate degree programs for students starting at 
community college/ We conclude by summarizing the advantages of applying FE models. 


2. Model Specifications 

Mincerian OLS Earnings Specifications 

Following Mincer (1974), a large volume of studies has estimated the labor market 
returns to college. These studies applied Ordinary Least Squares (OLS) regression models of 
earnings against highest level of attainment, controlling for work experience and other 
co variates: 


Y, = a + QEDUCt-k + 6 EXP t + cp EXP? + $X H + y Z t . k + s 


( 1 ) 


In equation (1), earnings Y (or log earnings) at quarter t are a function of: college 
education, EDUC, represented as, for example, awards or credits; work experience and its 
square, EXP and EXP 2 -, a vector of attributes of college (e.g., sector), X; and a vector of pre- 
college personal and ability-related characteristics, Z. The coefficient 9 represents an estimate of 
the earnings premium from education obtained in college (or the percentage increase if the 
variables are expressed in log form). 3 4 

The Mincerian OLS approach has several strengths. OLS results are easy to report and 
interpret. The dependent variable earnings is easily expressed (e.g., at age t, in a particular 
calendar year, r years after first-time enrollment, or s years after completion). Also, this 
expression is separate from inference as to how these average treatment effects apply to 
alternative populations or alternative measures of the dependent variable (Carneiro, Heckman, & 
Vytlacil, 2011). It is easy to check model specifications as well: preferably, the distribution of 


3 

Certificate awards and diploma awards have different meanings across states. For students who start at community 
college, the returns to bachelor’s degrees are complicated by transfer status. 

4 This earnings premium approximates to the return to college under strict assumptions about the cost of college. 
Our focus here is on the labor market gains from college and not the net returns. 
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covariates in treatment and control groups should be similar, and this can be tested 
straightforwardly; and goodness-of-fit can be clearly evaluated from /^-squared statistics. 5 

However, this Mincerian approach only yields an unbiased estimate of the gains from 
college if all other variables that are correlated with education and that determine earnings are 
included in the vectors X and Z. But most OLS models cannot control for all determinants and so 
are subject to omitted variable bias. If these omitted variables vary positively with education, the 
coefficient 9 will be an inflated estimate of the gains to education (Barrow & Malamud, 20 15). 6 
The main concern is that those who attend and complete college have higher ability; omission of 
ability will therefore bias the labor market gains upward. In most empirical studies where ability 
measures are available, 9 coefficients are lower but are typically still positive and statistically 
significant. 7 Hence, even as ability does affect earnings, it does not fully or straightforwardly 
explain the gains to postsecondary education (see also Belfield & Bailey, 2017). 

Other biases may be significant. For example, Webber (2016) identifies a sizeable 
influence of non-cognitive characteristics such as an individual’s locus of control and self- 
esteem. However, these characteristics may themselves be endogenous to either earnings 
(persons with high earnings may then report higher self-esteem) or education (those with degrees 
may report higher self-esteem, see de Araujo & Lagos, 2013). Another bias may be self- 
selection: students who value higher earnings correspondingly self-select into fields of study that 
yield higher earnings, or students enroll in education until the returns reach a threshold 
(Melguizo & Wolniak, 2012; Griliches, 1977). Overall, it is not certain that OLS approaches are 

o 

substantially biased in ways that make the coefficient estimates irrelevant for policy decisions. 

As a response to the possible biases associated with OLS approaches, a range of 
econometric techniques have developed. These techniques include instrumental variables 
estimation, natural experiments, and estimation using biases on unobservables (Altonji, Elder, & 


5 Goodness of fit for cross-sectional data on annual and quarterly earnings is approximately 0. 1-0.3 (Bahr, Dynarski, 
Jacob, Kreisman, Sosa, & Wiederspan, 2015; Liu et al., 2015; direct analysis of Current Population Survey [CPS]). 

6 Early analyses justified the use of Mincerian OLS estimation on the grounds that positive and negative biases 
offset each other (Griliches, 1977). Gelbach (2016) demonstrates how the specific influence of each additional 
covariate may be misinterpreted. Here we are interested in the most valid estimate of the returns to education, not 
the measured influence of each covariate. 

7 Using data from the 2000 follow-up of the National Education Longitudinal Survey (NELS), Marcotte (2010) finds 
that controlling for school quality and academic ability reduces returns to associate degrees by 19 percent for men, 
but that it increases them by 10 percent for women. Using the National Longitudinal Survey of Youth 1997 
(NLSY97), Scott-Clayton and Wen (2017) estimate the returns to associate degrees versus non-completers, which 
are 14 percent lower with full controls. Using NLSY79, Agan (2013) estimates OLS returns for associate degrees 
that are 30 percent lower with a full array of cognitive, non-cognitive, and family background characteristics. For 
Texas, Andrews, Li, and Lovenheim (2014) find that ability bias reduces the returns by approximately one-half and 
that college intentions reduces the returns for men (by 33 percent) but has no effect for women. For Virginia, 
controlling for demographic and personal characteristics increases the returns (Xu & Fletcher, 2016). For bachelor’s 
degrees in North Carolina, Liu et al. (2015) find the reverse: the controls increase returns for men but decrease them 
for women. In all cases, the coefficients on education are statistically significant. 

g 

In their detailed analysis, Carneiro et al. (2011, p. 2779) concluded that “[s]ome marginal expansions of schooling 
produce marginal gains that are well below average returns. For other policies associated with other marginal 
expansions, the marginal gains are substantial.” 
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Taber, 2005; Rouse, 2007). Discussion of these alternatives is beyond the scope of this paper. 9 
Here, we focus on a particular alternative technique that is increasingly being applied: fixed 
effects estimation. 

Fixed Effects Specifications 

The FE model specification for individual i in quarter of time t is given as: 


Y u = a + 0A WARDj, + fSENROLLED,, + yTIME it 

+ STIMEu x DEMOG + tpQ it + p, + q, + c„ (2) 


In equation (2), Y it is quarterly earnings from Unemployment Insurance (UI) records; 

A WARD is a binary indicator for possession of a degree, certificate or diploma in that quarter; 
ENROLLED refers to quarters in which the student is enrolled in college; TIME captures the 
effect of time on earnings; the interaction TIME x DEMOG captures variations in the effect of 
time with respect to individual characteristics; and Q is a vector of other terms included in some 
specifications (e.g., whether a student transfers to another college). The three remaining terms 
are: //„ which represents individual fixed effects (a dummy variable for each individual in the 
sample) and so controls for all individual fixed-over-time characteristics; rj t , which represents 
quarter (e.g., the first quarter of 201 1) fixed effects and so controls for quarter-specific 
macroeconomic circumstances; and s it , which is the error tenn. 

The advantage of the FE model of equation (2) is that it eliminates bias from time- 
invariant unobservable characteristics by differencing out person-specific mean earnings from 
observed quarterly earnings. 

However, there are several concerns with FE specifications. One concern is that results 
from FE specifications — and how these are interpreted — may be sensitive to model specification. 
Key temporal variables as represented by TIME can be modeled in various ways, to capture an 
individual’s age, economic conditions in each quarter, time enrolled, time since completion or 
exit, and economic events such as the Great Recession. Given that awards take different 
durations to complete (and that dropouts spend less time in college than award holders), these 
TIME variables and //, may affect how the coefficients on A WARD should be interpreted (see 
Couch & Placzek, 2010). For example, dropouts will have had a longer time in the labor market 
than award recipients; their mean value for time since completion will therefore be greater. 10 
Interpretation of FE results becomes even more challenging when TIME is interacted with 
demographic characteristics. Also, models vary with respect to the variables included in 

9 It is difficult to summarize the empirical literature using these alternatives, but the literature does not clearly 
establish that OLS estimates are strongly biased. For example, Cameiro et al. (2011, Table 5) estimate the returns to 
one year of college at 0.0836 using OLS and 0.0951 applying a linear instrumental variables approach. 

10 For analysis of the effects of more rapid time to completion on earnings gaps, see Flores-Lagunes and Light 
( 2010 ). 
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DEMOG and Q : specifications include gender, race/ethnicity, and socioeconomic status; they 
also include some quasi-fixed attributes (such as intent when entering college and first-semester 
GPA), and some time -varying attributes (e.g., credits per semester or cumulative credits per 
semester). Similarly, these variables are not independent of degree completion (students with 
only 10 cumulative credits cannot obtain an associate degree, for example). 11 Finally, education 
credentials may be modeled in various ways: for an individual who first obtains a certificate and 
subsequently an associate degree, the dummy variable for certificate may be toggled off (zero) 
when the degree is awarded or it may remain as a binary indicator. Thus far, no consistent 

12 

baseline functional form has been established; this makes comparisons across studies difficult. 

Another concern is that these FE models — because they are more data intensive — impose 
more restrictions on the composition of the estimation sample. FE models typically impose data 
and sampling constraints so that the earnings gains can be accurately identified. One constraint is 
that individuals should have work experience prior to enrolling in college (so that within-person 
latent labor market productivity can be controlled for). But many students progress straight from 
high school to college without relevant work experience, and some others may have worked in 
jobs that do not correspond to their future human capital and career aspirations. Also, FE models 
may be less valid for persons who enroll in college later in their career as a reaction to their 
otherwise low expected future earnings; selection into college is different for younger versus 
older workers (Jepsen & Montgomery, 2012). A related constraint is that the dataset must cover 
a long period of time, including the period before college, during college, and at least five years 
after exit from college. The influence of missing data (before, during, or after college) may 
therefore be uncertain. 

Moreover, FE specifications do not address all omitted variable bias. They do reduce 
omitted variable bias from fixed unobservable characteristics (e.g., latent ability). Also, they 
reduce endogeneity bias, i.e., bias because relatively low expected wages at y t induce college 
enrollment at y t +i (for all enrollments in college). Specifically, FE models require that there are 
no time-varying unobservable shocks correlated with educational attainment and labor market 
outcomes that differentially affect college completion. One particular shock — the student’s 
realized performance in college and its implications for future earnings — is likely to be highly 
salient. Indeed, given low completion rates across the two-year sector, it seems likely that many 
students do change their effort and motivations after they first enroll in college. 

This concern raises questions as to the validity of FE estimates and the importance of 
time trend specifications. Although FE estimation has been extensively used in program 
evaluation, its application to education-eamings gradients may not be valid. As emphasized by 

1 1 Students who work while enrolled may experience slower earnings growth than students who do not work and 
secure full-time work only after graduation. This disparity may be greater for students who work in jobs that 

complement their studies. 

12 

" For example, in examining returns across college students in Texas, Andrews et al. (2014, 2016) estimate the 
returns based on individual-level residuals. First, earnings are regressed on year, quarter, and high school cohort 
indicators; the residuals from this equation are used as the dependent variable for the FE model. As discussed below, 
recent statewide models differ from one of the first studies by Jacobson et al. (2005). 
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Sobel (2012), there are issues over how coefficients should be interpreted given the model 
specification. 

Figure 1, showing earnings data from Ohio from Minaya and Scott-Clayton (2017), 
illustrates the challenge for FE estimation. Figure 1 shows the paths of earnings for associate 
degree completers and for persons who do not complete an award. These paths are very different 
in several respects. Before entry, the earnings paths are similar, but on entry to college the 
average earnings of completers are much lower than those of non-completers; this deficit may be 
an endogenous response to focusing on college rather than work because there is a higher 
probability of completing the degree. During college, degree recipients have much lower 
earnings: they are specializing in college more intensively than non-completers. Finally, post- 
college the earnings gap grows significantly for associate degree recipients. By including pre-, 
during, and post-college periods, FE models are attempting to identify the returns to education 
from a model of this entire trajectory. 


Figure 1: Quarterly Earnings by Quarter Since First Entry (Women, Ohio) 



FE models are identifying gains in earnings from the post-college gap in earnings. These 
models typically have several implicit assumptions. First, there should be immediate impacts of 
college awards on earnings (i.e., it is appropriate to include all quarters after the college award as 
reflecting degree-level labor market skills); this seems unlikely given that graduates are 
searching for work that best fits their award and that this search may take time. Second, the 
earnings at time t should not dependent on earnings at time t - 1; this assumption may also be 
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questioned given that the profiles show individuals willing to sacrifice current income for greater 
future income (and having been unemployed in previous quarters is almost certainly going to 
influence wages in subsequent quarters). Finally, education at time t should be assumed to be 
independent of earnings at time t - 1; yet, it seems plausible that, if earnings at time t - 1 are 
unexpectedly high, an individual will not go back to college to increase earnings. More 
generally, ability, motivation, and latent productivity are unlikely to be constant over time: 

13 

students who discover that they can do well in college may be motivated to study harder. 
Therefore, following Dynarski, Jacob, and Kreisman (2016), we investigate time trend effects 
across FE models in detail. 


3. Testing the Robustness of Earnings Equations 

To test for the robustness of FE models we distinguish two sets of empirical issues: those 
related to functional form and those related to sampling. We recognize that identification 
challenges can be addressed in multiple ways. For example, in testing for robustness with respect 
to the ages of students, the specification might include more age-related covariates or divide the 
sample into age groups for subgroup estimation. Also, we recognize that the results will vary 
across subgroups of students (e.g., by race/ethnicity); our focus in this review is not on the 
heterogeneity of results across these subgroups. 

We perform a full set of sample and functional fonn sensitivity tests for the FE model 
using data from students who first enrolled in the North Carolina Community College System 
(NCCCS) in 2002-04. Full details on this dataset are given in Liu et al. (2015). We begin by 
estimating the expected gains for an associate degree completer over a non-completer from a 
baseline model. This baseline excludes persons with zero earnings. 

The baseline estimates for North Carolina over non-completers are $1,161 for men and 
$1,932 for women in quarters when the individual’s highest award is an associate degree. 
Relative to average quarterly earnings of $7,983 (men) and $6,478 (women) in the final year of 
follow-up, this return is statistically significant; it equates to an earnings premium of 15 percent 
for men and 30 percent for women. This estimated gain is precisely estimated: the standard error 
is approximately $50. We then compare this baseline estimate to alternative samples and 
functional forms. 

In addition, we summarize the range of robustness checks and sensitivity tests applied in 
other studies that rely on large-scale transcript datasets matched to UI earnings. For a review of 
the overall results from these studies, see Belfield and Bailey (2017). These studies apply fixed 
effects models in different ways and then apply sensitivity checks in correspondingly different 


13 Also, there are issues of interpretation as to how well the data fit the model based on goodness-of-fit tests. There 
are goodness-of-fit statistics for FE models, but these are more difficult to interpret relative to OLS R-squared 
statistics, not least because model specifications vary significantly. 
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ways — so the robustness of the results across studies is not easy to summarize. Therefore, instead 
of calculating differences in results across specifications, we summarize the magnitude and sign 
of each robustness check. 

Finally, we compare the FE results with OLS specifications that attempt to identify an 
equivalent construct — the expected steady-state earnings gains from completing an associate 
degree. To make this comparison, we need to specify a comparable estimation approach. As for 
our main test, we calculate estimates directly using administrative data from North Carolina; we 
also summarize results from the available specifications in related studies from other states. 
Given the motivation for FE models — that unobservable characteristics positively associated 
with educational attainment are positively associated with earnings — we expect that the OLS 
estimates will be considerably higher than the FE estimates. Again, however, we caution that for 
the comparison to be valid the two estimates must be measuring the same construct. 


4. Results From Sensitivity Tests 

FE Models: Sensitivity to Sample Inclusion Criteria 

We investigate how sensitive the baseline result is to variations in sample inclusion 
criteria. The baseline estimate is given in column 1 of Table 1, and the alternatives are shown in 
columns 2-7. Overall, Table 1 shows that the returns to associate degree holders compared to 
community college students who do not complete an award are large and always statistically 
significant across the sample specifications. 

Time window. FE models require longitudinal data over a long window of time. For 
example, the North Carolina estimates are based on transcript information for students who 
entered college in 2003, ostensibly for a two-year degree. The earnings data are from the period 
from the 1990s up to 2013; that is, there is more than a decade’s worth of earnings data. Even 
this coverage may be too brief: a student entering college in 2003 who earns a bachelor’s degree 
may not fully enter the labor market until 2009; this would yield at most four years of earnings 
data. Therefore, it is important to investigate whether the time windows for analysis matter and 
how much. 14 

Overall, the evidence shows that the time frame for analysis does matter in two respects. 
First, college students do not immediately get full-time employment in a job matching their 
skills. For displaced workers in Washington State, Jacobson et al. (2005, Table 2) estimate that 
first-quarter earnings “gains” are negative for trained workers. Second, the growth in earnings 
for associate degree holders is steeper than for non-completers. Almost all studies identify this 
steeper trajectory (see, e.g., Jaggars & Xu, 2016). Both these factors mean that results from FE 
models will be sensitive to the number of quarters available in the dataset. 

14 OLS estimation only requires the year of earnings and an indicator for college education in the past. 
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Table 1: Quarterly Earnings Gain Fixed Effects Estimation: Sample Specifications for North Carolina 



( 1 ) 

Baseline 

(2) 

Excluding 
Low Wage 

( 3 ) 

Excluding 
Age < 20 

( 4 ) 

Excluding 

Bachelor’s 

Degrees 

( 5 ) 

Excluding 

Transfers 

( 6 ) 

Including 
Zero Wage 

( 7 ) 

Shorter 

Window 

Men 

Associate degree vs. no award 

1,161 

1,152 

1,140 

1,159 

1,432 

1,513 

1,121 


(43)*** 

(43)*** 

(53)*** 

(45)*** 

(56)*** 

(42)*** 

(49)*** 

Coefficient vs. baseline 


-1% 

-2% 

0% 

+23% 

+30% 

-3% 

iCsquared 

0.159 

0.151 

0.327 

0.129 

0.124 

0.073 

0.148 

N persons 

93,430 

93,164 

45,637 

81,272 

46,555 

104,353 

93,016 

N quarters 

1,992,497 

1,967,562 

931,067 

1,749,367 

1,164,127 

3,618,605 

1,779,409 

Women 

Associate degree vs. no award 

1,932 

1,920 

2,114 

1,991 

2,254 

2,291 

1,815 


(30)*** 

(30)*** 

(45)*** 

(31)*** 

(40)*** 

(29)* * * 

(34)*** 

Coefficient vs. baseline 


-1% 

+9% 

+3% 

+17% 

+19% 

-6% 

iCsquared 

0.191 

0.183 

0.359 

0.141 

0.135 

0.110 

0.154 

N persons 

141,683 

141,356 

58,358 

121,919 

68,373 

149,470 

140,826 

N quarters 

3,255,979 

3,214,126 

1,2 77,737 

2,820,617 

1,780,395 

5,202,582 

2,538,107 


Note. All first-time-in-college NCCCS students (academic years 2002-03 to 2004-05) with earnings up to 2012. Model as per equation (1): Award; Dip4; 
Quarter; QuarterSqd; Race*Quarter; Age25* Quarter; Race*QuarterSqd; and Age25*QuarterSqd. Constant term included. Award status toggled for highest 
award. Quarter is calendar quarter (e.g., 2002Q3). Low wage is < $750 per quarter pre-college. Transfer is quarter when student is in a transfer college. Shorter 
window is two years shorter follow-up. Robust standard errors reported in parentheses. 

*p < .1. **p < .05. ***p < .01. 
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Table 2: Quarterly Earnings Gain Fixed Effects Estimation: Model Specifications for North Carolina 



( 1 ) 

Baseline 

( 2 ) 

Cubic 

( 3 ) 

Extra 

Time 

Variables 

( 4 ) 

Dips 

Precollege 

( 5 ) 

Post- 

College 

Dummy 

( 6 ) 

Transfer 

Dummy 

( 7 ) 

Award x 
Enrolled 

( 8 ) 

Award 

Hold 

Men 

Associate degree 

1,161 

1,152 

1,161 

1,152 

1,174 

1,166 

1,260 

848 


(43)*** 

(43)*** 

(43)*** 

( 43 )*** 

(43)*** 

(43)*** 

( 45 )*** 

(42)*** 

Coefficient vs. baseline 


- 1 % 

0 % 

- 1 % 

1 % 

0 % 

9% 

-27% 

.R-squared 

0.159 

0.160 

0.159 

0.159 

0.161 

0.160 

0.159 

0.158 

N persons 

93,430 

93,430 

93,430 

93,430 

93,430 

93,430 

93,430 

93,430 

N quarters 

1,992,497 

1,992,497 

1,992,497 

1,992,497 

1,992,497 

1,992,497 

1,992,497 

1,992,497 

Women 

Associate degree 

1,932 

1,923 

1,932 

1,927 

1,937 

1,928 

2,053 

1,583 


(30)*** 

(30)*** 

(30)*** 

(30)*** 

(30)*** 

(30)*** 

(31)*** 

(30)*** 

Coefficient vs. baseline 


0 % 

0 % 

0 % 

1 % 

0 % 

7% 

-18% 

iCsquared 

0.191 

0.193 

0.191 

0.191 

0.193 

0.191 

0.192 

0.188 

N persons 

141,683 

141,683 

141,683 

141,683 

141,683 

141,683 

141,683 

141,683 

N quarters 

3,255,979 

3,255,979 

3,255,979 

3,255,979 

3,255,979 

3,255,979 

3,255,979 

3,255,979 


Note. All first-time-in-college NCCCS students (academic years 2002-03 to 2004-05). Model (1): as per equation (1): Award; Dip4; Quarter; QuarterSqd; 
Race*Quarter; Age25*Quarter; Race*QuarterSqd; and Age25*QuarterSqd. Award status toggled for highest award. Constant term included. Quarter is calendar 
quarter (e.g., 2002 Q3). Model (2): Baseline and QuarterCubed; Race*QuarterCubed; Age25*QuarterCubed; and Age25*QuarterCubed. Model (3): Baseline and 
Seasons (3). Model (4): Baseline and Indicator (no longer in college). Model (5): Baseline except Dip4 replaced by Indicators (two quarters before enrollment). 
Model (6): Baseline and Indicator (Transfer student) and Interaction (Transfer x enrolled). Model (7): Baseline and Interactions (award x enrolled). Model (8): 
Baseline except Award indicator held for all quarters when award held regardless of other awards. Robust standard errors reported in parentheses. 

*p < .1. **p < .05. ***p < .01. 
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Table 3: Robustness Checks on Returns to Associate Degrees (Several State Studies) 



Washington 

Michigan 

Kentucky 

California 

Colorado 

Ohio 


Female 

Male 

Female 

Male 

Female 

Male 

Pooled 

Pooled 

Pooled 

Baseline FE including time trend 

6.5% 

2.1% 

$2,363 

$1,484 

$2,346 

$1,441 

$1,650 

$2,200 

$2,439 

Sample selection 

Age 25+ on entry; worked pre-entry 

— 

— 








Age 20-50 







+++ 


n.d. 

Excluding younger persons 





+ 

n.d. 




Including younger persons 



n.d. 

n.d. 






Short window (years post-college) 



n.d. 

n.d. 



- 


— 

Excluding enrolled in college five 
years later 

- 

+ 



+ 

+ 

+++ 

n.d. 


Adjusting for enrollment of non- 
completers 








n.d. 


Excluding zero credits in college 



n.d. 

- 



n.d. 



Only employed 



+ 

+ 






Including missing wages pre-entry 

+ 

+ 








Including strong labor market 
attachment 



n.d. 

n.d. 




+ 


Including weak labor market 
attachment 





n.d. 

+ 




Specifications 

Baseline excluding time 
trends/interactions 

+++ 

+++ 

n.d. 

n.d. 



n.d. 

n.d. 


Difference estimator 



- 

n.d. 




- 


Adjusting for Ashenfelter Dip (1) 

n.d. 

n.d. 



n.d. 

n.d. 

n.d. 

- 


Adjusting for Ashenfelter Dip (2) 

n.d. 

+ 

n.d. 

n.d. 

n.d. 

- 





Note. See Appendix Table A1 for full results. Sources: Dadgar and Trimble (2015) (Washington); Bahr et al. (2015) (Michigan); Jepsen et al. (2014) (Kentucky); 
Bahr (2016) (California); Turner (2016) (Colorado); Minaya & Scott-Clayton (2017) (Ohio). 

n.d. = estimate within 5% of baseline estimate. 

+/- = estimate between 5-10% of baseline estimate. 

++/— = estimate between 10-20% of baseline estimate. 

+++/ — = estimate more than 20% different from baseline estimate. 
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The size of the distortion from having too brief a window is unclear. For Colorado, 

Turner (2016) finds that the earnings gains in the initial quarters after exit from college are a 
good approximation to the long-run benefits; earnings gains are also robust with a shorter 
window for community college students in Michigan (see Table 3 above). For North Carolina, 
column 7 of Table 1 shows only a modest negative effect from truncating the follow-up period. 
Returns are 3 percent lower (men) and 6 percent lower (women) if the follow-up is shortened by 
two years (to a maximum of seven years since first enrollment). 15 However, the analysis in Table 
1 compares a moderate time frame with a short time frame. The clearest test comes from analysis 
of Ohio students: Minaya and Scott-Clayton (2017) calculate that the returns are 25 percent 
higher if the window of analysis is extended from 5 to 8 years of earnings data post-college 
(Minaya & Scott-Clayton, 2017, final column of Table 2). Analyses with earnings data of less 
than 5 years therefore appear to be significantly underestimating the long-run returns to college. 

Pre-college work experience. FE models identify the earnings gains from a degree based 
on deviations from individual-level earnings over the period pre-college and during college. For 
persons who enter community college almost immediately after high school, any pre-college 
earnings are unlikely to be a good guide to their expected earnings post-college. It seems likely 
that, for these persons, th e AWARD coefficient will be biased upward, capturing the value of the 
degree and employment in a job that corresponds to their latent productivity. By contrast, for 
older students who had significant work experience prior to college, the A WARD coefficient 
should more accurately reflect the direct change in skills — and hence earnings — from having a 
degree. 

One test is to see how returns vary if the sample is restricted to older workers. Applying 
an age restriction does reduce the sample size for analysis. In Washington State, 49 percent of 
community college students are aged 19 or under at first enrollment (Dadgar & Trimble, 2014), 
so the analysis sample is reduced by 5 1 percent; in Michigan, the analysis sample is reduced by 
20 percent (Bahr et ah, 2015); and in North Carolina the sample is reduced by 5 1-58 percent. 

However, the effect on th e AWARD coefficient is not affected significantly. Column 3 of 
Table 1 shows results for North Carolina: excluding persons aged under 20 has a trivial impact 
for men and a modestly positive impact for women. Given that the sample is reduced by over 
half, the overall effect is surprisingly small. As summarized in Table 3, studies from other states 
find a mixed pattern. For Kentucky, excluding persons aged under 20 has a small effect: it 
reduces returns negligibly for men but increases them slightly for women. Similarly, in Ohio, the 
differential effect of age is very small. In Washington (in some specifications) returns to 
associate degree holders are substantially lower if younger persons are excluded (and this is also 
the case for some specifications in Michigan, not reported in Table 3). However, earnings gains 
are higher in California for older workers (aged 20 or over on first entry); and in Virginia (Xu & 

This decline was also found by Liu et al. (2015) using OLS estimations and the 2002-3 cohort. But the decline in 
earnings is non-linear: returns are 13 percent (men) and 15 percent (women) lower if we shorten the follow-up by 
three years. Also, truncating the follow-up by one year reduces the returns by only 2 percent (men) and 3 percent 
(women). 
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Fletcher, 2016) returns to an associate degree are higher for those aged over 25 (not reported in 
Table 3). A confounding factor is that wage dispersion increases with age: point estimates of 
returns to awards are less precise for older workers. 

In addition, we re-estimate equation (1) by excluding persons with zero (or very low) pre- 
college earnings. This is a direct test of the influence of pre-college work experience. As shown 
in column 2 of Table 1, excluding persons with low pre-college wages has a trivial impact on the 
earnings gains. Under various specifications, this result has also been found in Michigan, 
Kentucky, and Washington states (Bahr et ah, 2015; Dadgar & Trimble, 2014; Jepsen, Troske, & 
Coomes, 2014). 16 

Zero earnings. FE models rely on data from across at least one decade; results for 
individuals with high proportions of missing data or intermittent labor market participation over 
this period may be biased. Data may be missing for many reasons (including incomplete records, 
out-of-state migration, zero earnings coded as missing, and exemption from UI coverage). These 
reasons may be collinear with education: if degree completion increases labor force participation 
rates, then assigning zeroes as missing data will underestimate the returns to college. 17 

As shown in column 1 of Table 1, the baseline model for North Carolina excludes 
quarters with no earnings data. Including all quarters for each person and assigning missing 
earnings as zeroes does make a substantial difference. As shown in column 6, the returns to an 
associate degree are 30 percent higher among men and 19 percent higher among women if we 
include zero earnings compared to baseline. Evidence from other states is less emphatic (Table 
3): adjusting for labor market attachment has a small positive effect on estimated returns. These 
weaker results are surprising given the positive association between education and labor market 
participation. 18 

A related consideration is the sample of students who remain in college beyond a 
reasonable time frame (e.g., greater than 150 percent of “normal” time to completion). Durations 
in college have been growing longer (Bound, Lovenheim, & Turner, 2010). These persons may 
still be in college because they have not been able to find work, i.e., their earnings are effectively 
zero. Also, they may have insufficient years of work from which to identify differences in the 
growth in earnings post-college. As a robustness check, these persons may be excluded from the 
sample. As shown in Table 3, however, this exclusion criterion has inconsistent impacts: in 
Washington, Kentucky, and Colorado, there is a very slight difference in the returns to an 


16 For Washington, Dadgar and Trimble (2014) report coefficients for associate degrees of 0.102 (men) and 0.088 
(women) with a sample only for those with prior earnings data; including all persons yields similar coefficients of 
0.114 (men) and 0.074 (women) (M. Trimble, personal communication, July 20, 2015). 

17 It is not possible to distinguish quarters with zero earnings from quarters with missing earnings in the North 
Carolina dataset. Some preferred specifications do include persons with zero earnings (e.g., those by Dadgar & 
Trimble, 2015; Jepsen et ah, 2014); others exclude zero earnings (e.g., Bahr et ah, 2015; Jaggars et ah, 2016; Liu et 
ah, 2015). 

18 In Michigan, returns to associate degrees appear lower when the sample is only those who are employed. Using 
NLSY97, estimates for worker-only samples yield gains from associate degrees that are 14 percent lower than 
estimates for all persons (Scott-Clayton & Wen, 2017, Table 3). 
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associate degree; in California, returns are significantly higher when the sample excludes persons 
who are enrolled in community college for more than five years. Overall, there is no strong 
evidence that the trend toward longer durations in college has an impact on the estimation of 
returns. 

Student transfers. Another concern is missing student-level data on college attainment. 
National Student Clearinghouse transfer data do not cover all colleges that students might 
transfer to, and some colleges do not submit reports on where their students had previously 
enrolled (see Dynarski, Hemelt, & Hyman, 2015). Overall, lack of coverage in the transfer data 
affects only a small subset of students: almost all large public institutions report their transfers to 
the National Student Clearinghouse (and coverage is increasing over time). But lack of coverage 
may affect clusters of students in for-profit colleges, and coverage does vary by state: some 
states cover only two thirds of relevant postsecondary institutions. Uncovered students who 
transfer to other colleges will have more — unobserved — postsecondary human capital (either 
credits or a credential); the returns to observed degrees may therefore be biased upward when 
transfer information is missing. Another group for whom data is missing is reverse transfer 
students, those who transfer from a four-year to a two-year college to earn an associate degree; 
these students are not typically included in datasets that are collated based on first-semester 
enrollment at a college. 

The results in column 5 of Table 1 show that, for North Carolina, excluding transfer 
students significantly increases the estimated returns to an associate degree. 19 When transfer 
students are excluded, the returns to an associate degree are increased by 23 and 17 percent for 
men and women respectively. This relationship also holds for Kentucky: Jepsen et al. (2014) 
identify significant differences depending on whether or not transfer students are included. 
Notably, including transfer students decreases estimated returns because the time window of 
earnings data is not long enough. (We are not aware of robustness checks on transfer students in 
other state studies). However, for North Carolina at least, the returns to an associate degree are 
not substantially affected if students who subsequently earn a bachelor’s degree are excluded. By 
inference, students who transfer but do not complete a bachelor’s degree have significantly lower 
earnings, and results may be sensitive to the size of this group. Looking across all transfer 

students, though, there is no evidence that this group, regardless of whether they complete any 

20 

award, have higher ability than students who earn an associate degree." 

Overall, results are sensitive to whether transfer students are included in the sample. 
Modeling the pathways of transfer students therefore appears to be an important concern when 
estimating the returns to college, although the reasons for this remain unclear. 


19 Relevant to our comparisons above, North Carolina data has near complete data coverage at the student-level 

(over 90 percent, Dynarski et al., 2015, Table 2). 

20 

Using NLSY79, Agan (2013, Table 3) estimates that for associate degree recipients who transfer, the private 
internal rate of return is 1 1 and 16 percent for men and women, respectively, if they earn a bachelor’s degree, and 3 1 
and 25 percent if they do not. For those with an associate degree who do not transfer, the internal rate of return is 22 
and 32 percent. 
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FE Models: Sensitivity to Model Specification 

Next we consider functional form differences within the fixed effects specification. For 
North Carolina, variations in returns across alternative functional forms are reported in Table 2 
above; the alternatives are again compared to the baseline estimate, given in column 1 . Also, for 
other states, variations are summarized in the bottom panel of Table 3 (above) relative to the 
baseline in the first row of Table 3 (also see Appendix Table Al). As with sample restrictions, 
some alternative functional fonn specifications have a small influence on the estimated returns. 
There are some important exceptions, although in these cases we must be cautious because the 
interpretation to be placed on the coefficient has changed. 

Ashenfelter dip. Earnings show a clear drop (Ashenfelter dip) before students enter 
college, raising the possibility that college enrollment is endogenous to future earnings paths. 
Conventionally, the Ashenfelter dip refers to the circumstance in which persons who experience 
negative earnings shocks are more likely to subsequently enroll in college. All the statewide 
studies do show earnings dips prior to college entry. However, this dip may be endogenous, i.e., 
the student may be giving up a job in order to enroll more intensively in college. As shown in 
Appendix Figure Al, the Ashenfelter dip is much larger for students who eventually complete an 
associate degree relative to those who do not complete a degree. Students who complete a degree 
may be voluntarily trading off work for college. 

In FE models, attempts to address the Ashenfelter dip do not change the estimated gains 
from completing an associate degree. As shown in column 4 of Table 2, the point estimate of 
returns changes by only 1 percent. As summarized in Table 3, the studies have attempted six 
alternative specifications to adjust for the Ashenfelter dip: only two yield slightly different 
estimates, and these differences are not consistent in sign. Overall, functional forms to model 
pre-entry behavior do not appear to be influential. 

Modeling earnings trajectories. As discussed above, a key challenge is how to model 
time trends of earnings growth for degree holders relative to a baseline earnings growth. It seems 

plausible that individuals who obtain degrees are on different earnings trajectories both before 

2 1 

and after college. 

A simple approach would be to include more varied specifications of independent TIME 
variables. As shown in Table 2 (columns 2-6), alternative specifications that have little impact 
on the returns to awards are: including cubic time trends; extra time variables (season, year) and 
inclusion of indicators for quarters either after college or as a transfer student. These alternatives 
affect the coefficients by less than 2 percent. These findings correspond to results from other 
states (see Table 3).' For Kentucky, Jepsen et al. (2014) find that adding time trends reduces 
coefficients by approximately 4 percent. Thus, more complex specifications to account for time 


Time trends while in college appear less influential (see Jaggars & Xu, 2016). For Ohio, Bettinger and Soliz 
(2016, Tables 3 and 4) estimate returns that are statistically equivalent for students who work intensively when in 

college. 

22 

For Connecticut, Crouch, and Placzek (2010) find slightly higher wage growth when time trends are included. 
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do not affect the results. However, estimated returns are significantly different if award receipt is 
interacted with quarters of enrollment. As shown in column 7 of Table 2, this interaction 
increases the returns to an associate degree by 9 and 7 percent for men and women respectively. 
This is further evidence that the returns may be increasing with time since exit from college. 

In analysis for Michigan, Dynarski et al. (2016) investigate a range of alternative 
estimations to more accurately capture time trend differences between degree completers and 
non-completers. One approach is to control for earnings growth by including a measure of 
lagged earnings to account for initial productivity. This inclusion has a significant impact on 
returns: the gains from an associate degree fall from a baseline of $1,129 to $394 when lagged 
earnings are included (Dynarski et al., 2016, Table 3). A more comprehensive model 
specification includes an individual time trend; this captures differences in growth trends across 
individuals. However, returns to an associate degree increase by 17 percent when individual time 
trends are included (Dynarski et al., 2016, Table 4). Lastly, in looking at pre-college differential 
trends in earnings, Dynarski et al. (2016, Figure 1) show distinct earnings paths for students who 
ultimately earn an associate degree. Specifically, these students experience relatively sharp 
declines in earnings before entering college; for those who do not earn an award, the pre-college 
earnings trajectory is consistently flat. This finding is more evidence that a simple FE model is 
unlikely to account for variations in earnings trajectories across student groups. That is, FE 
models need to account for pre-college dips in earnings, earnings trends when in college, and 
differential earnings growth after exiting college. However, the implications of this finding are 
less clear (in part because, as noted above, direct attempts to address Ashenfelter dip effects have 
little influence). If associate degree holders are experiencing exogenous and relatively sharp 
declines in earnings before they enter college, this suggests that they are less advantaged than 
other student groups and that the post-college earnings gains are biased downward. Alternatively, 
a decline in earnings before entering college may indicate that these students are more motivated 

23 

to complete a degree. 

Analysis using highest award. In some FE specifications the A WARD variable is 
toggled on (equals one) for the first award and then toggled off (equals zero) when the individual 
receives a higher award. The coefficient on each award is therefore interpreted as the return only 
when that award is the highest award the individual possesses at that time. By contrast, the 
alternative specification is to keep the AWARD variable toggled on for the entire time period 
after each award has been received. 

There is a significant difference in the results depending on which specification is used. 
For North Carolina, column 8 of Table 2 shows the difference in results if the award is toggled 
on permanently post-award compared to the specification where the award is toggled on and then 
off. The former specification (always toggled on) yields returns to associate degrees that are 

23 

In their modeling of displaced workers in Washington state, Jacobson et al. (2005) include as an independent 
variable the reciprocal of quarters since exiting college (see also Turner, 2016). This approach affirms that degree 
holders have different earnings trajectories over non-completers. However, the magnitude of the effect cannot be 
easily interpreted relative to the baseline model. 
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sharply reduced (by 27 and 18 percent for men and women respectively). However, the 
interpretation of th e AWARD coefficient is different across the two specifications — the 
coefficients need to be added up, and this infonnation is not clearly available — so we cannot 
conclude that FE models are not robust. 

Overall Sensitivity 

Looking across the models, we identify two important sampling issues (transfer and zero 
wages), two salient functional form issues with implications for interpreting the returns to 
college (award/enrollment interactions and award-holding), and one important specification issue 
(the type of time-trend estimator). 

For North Carolina, we can calculate the upper and lower bounds for estimates of the 
returns to college, adjusting for sampling and functional fonn issues. A lower bound estimate 
arises by applying the model where award-holding is toggled on/off: with this model the 
quarterly gains from an associate degree are conservatively estimated at $848 (men) and $1,538 
(women) above mean quarterly earnings. This estimate is a premium of 1 1 and 24 percent over 
earnings in the last year of follow-up. One upper bound estimate arises from a model including 
zero earners: with this model the quarterly earnings gains are $1,513 (men) and $2,291 (women). 
However, we can pool all modeling assumptions that upwardly effect the earnings estimates; that 
is, we can estimate the returns across all persons (zero and non-zero earnings) who do not 
transfer to a four-year college, with award interacted with enrollment status in each quarter. With 
this maximal upper bound, the returns to an associate degree increase to $2,007 (men) and 
$2,917 (women), which amounts to a premium of 25 and 45 percent over mean quarterly 
earnings in the last year of follow-up. 

Comparison of FE Model and OLS Model Estimates 

We now compare results from FE models with those from OLS estimates. The intent is to 
estimate the “steady-state” quarterly earnings gain from completion of an associate degree over 
non-completion. The estimates are calculated as the best estimate from each state study 
(approximately 8-10 years after first-time enrollment in college). However, they are not from 
directly equivalent samples, and they do not account for salient variations in sampling or 
functional form. 24 

Comparative estimates across several state analyses are summarized in Table 4. There are 
some modest differences in estimated returns. There is no clear indication that OLS estimates are 
substantially inflated over FE estimates. In five states (Michigan, Kentucky, Virginia, North 
Carolina, and Ohio), the gap between OLS and FE estimates is modest (less than a 15 percent 
difference); in two states (Michigan for women and Kentucky), the FE estimate is actually 

24 

Also, our comparisons assume that the coefficients from FE models are correctly interpreted with respect to time 
trend effects. As discussed above, we cannot be certain that the influences of all TIME covariates are correctly 
considered when interpreting th e AWARD coefficients. 
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greater than the OLS estimate. Washington is the only state that clearly shows OLS estimates 
substantially higher than FE estimates. 

Table 5 explores the differences between OLS and FE models in more detail with direct 
comparisons for North Carolina. These estimates use identical samples and are directly 
comparable (to be conservative, we apply a very parsimonious OLS specification controlling for 
race/ethnicity, age on entry, and months of work [squared]). Broadly, the results are similar. For 
three of these models (baseline, including zeroes, and excluding transfers), the coefficients are 
significantly higher using the FE model than the OLS model. For the fourth model (award held), 
the coefficients for the OLS model and the FE model are very similar (within 2-5 percent). We 
emphasize that the OLS model is very parsimonious and does not control for characteristics such 
as ability, which we would expect to lead to inflated OLS coefficients. However, the robustness 
and consistency of results between FE and OLS models is not surprising given that, as noted 
above, other studies that have sought to address ability bias have not found substantially different 
results. 


Table 4: Quarterly Earnings Gain Estimations for Associate Degree Completers Over Non- 
Completers: OLS and FE Models (Several State Studies) 



OLS 

Estimation 

FE 

Estimation 

(FE-OLS)/OLS 

Pooled 

Virginia 

$2,989 

$2,881 

-4% 

Ohio 

$2,439 

$2,313 

-5% 

Women 

Michigan a 

0.319 

0.327 

+2% 

Kentucky 

$2,290 

$2,363 

+3% 

North Carolina 

$2,136 

$1,907 

-11% 

Washington 

$1,051 

$600 

-43% 

Men 

Michigan a 

0.272 

0.118 

-14% 

Kentucky 

$1,349 

$1,484 

+10% 

North Carolina 

$1,115 

$1,113 

0% 

Washington 

$914 

$400 

-56% 


Note. Coefficient amounts not adjusted to constant dollars across states. Sources: Dadgar and Trimble (2015) 
(Washington); Bahr et al. (2015) (Michigan); Jepsen et al. (2014) (Kentucky); Bahr (2016) (California); Minaya & 
Scott-Clayton (2017) (Ohio). Jaggars and Xu (2016) (Virginia). 

a Coefficients on ln(earnings) regressions; percent gap calculated based on mean quarterly earnings. 
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Table 5: Quarterly Earnings Gain Estimations: OLS and FE Models for North Carolina 



(1) 

Baseline 

(2) 

Including Zero 
Earnings 

(3) 

Excluding Transfers 

(4) 

Award Hold 


OLS 

FE 

OLS 

FE 

OLS 

FE 

OLS 

FE 

Men 

Associate degree 

1,206 

1,579 

927 

1,509 

1,435 

1,928 

1,206 

1,170 


(61)*** 

(46)*** 

(43)*** 

(42)*** 

(75)*** 

(59)*** 

(61)*** 

(45)*** 

R- squared 

0.243 

0.176 

0.456 

0.074 

0.267 

0.141 

0.243 

0.176 

N persons 

N quarters 

61,233 

61,233 

2,125,043 

103,479 

103,479 

3,601,101 

36,412 

36,412 

1,273,116 

61,233 

61,233 

2,125,043 

Women 

Associate degree 

2,120 

2,430 

1,809 

2,292 

2,559 

2,820 

2,120 

2,018 


(34)*** 

(32)*** 

(26)*** 

(29)*** 

(39)*** 

(40)*** 

(34)*** 

(31)*** 

R-squared 

0.228 

0.217 

0.414 

0.110 

0.250 

0.174 

0.228 

0.214 

N persons 

N quarters 

98,937 

98,937 

3,449,282 

148,117 

148,117 

5,174,049 

55,049 

0.250 

55,049 

1,939,276 

98,937 

98,937 

3,449,282 


Note. All first-time-in-college NCCCS students (academic years 2002-03 to 2004-05). OLS earnings measured as average quarterly earnings in 2012. FE 
earnings quarterly from 2002-12. All earnings in 2010 dollars. OLS estimation controls for months of work experience (squared), calendar year (2), and 
race/ethnicity (3). FE estimation using baseline model as per equation (1) in Table 1. Standard errors in parentheses. 

*** p < 0 . 01 . 
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Table 5 does show that OLS models do identify the returns to awards less precisely 
(standard errors are wider). But in all cases the returns are statistically significantly different 
from zero at the 1 percent level. Also, the explained variation in earnings appears slightly higher 
for OLS models than FE models." With respect to efficiency and goodness of fit, FE models do 
not appear to have a clear advantage over OLS models. 

Finally, the results from Table 5 help reconcile this equivalence with the main finding 
from this review of FE results. The main finding is that results from FE models do vary with 
respect to some sample selections and functional forms; yet, FE results and OLS results are not 
significantly discrepant. As shown in Table 5, OLS results also vary with respect to sample 
selections. Thus, OLS results stay generally equivalent to FE results. 


5. Other Empirical Problems 

Overall, the results from across the FE specifications and relative to OLS specifications 
appear robust and consistent. Here we consider if other empirical problems are more salient. One 
concern is missing earnings data. The Unemployment Insurance (UI) earnings data for a given 
state typically do not include all workers; they exclude independent contractors, military 
personnel, some federal personnel, and those working in the infonnal sector. Separately, there 
may also be migration out of state; UI records typically do not capture out-of-state earnings. In 
practice, around 10 percent of all workers do not have reported earnings (zero or a positive 
amount) over the full period of analysis." It is unclear how salient this missing data bias is; the 
available evidence suggests it is probably modest. In a direct comparison with survey data, 
Wallace and Haveman (2007) found that UI data yielded lower returns to college and that the 
biases were smaller for those with steady jobs (such as college-educated workers). Recently, the 
U.S. Department of Education examined the bias from using UI data compared to tax return data 
(Executive Office of the President of the U.S., 2015). The difference in college-level average 

7R 

earnings across the two sources of data was less than 1 percent." 

The studies analyzed here do not address the problem of missing earnings due to out- 

7Q 

migration." If students who migrate out of state are those with the most job offers, then there 

25 

Goodness-of-fit tests are harder to interpret and have not been reported consistently. For example, Liu et al. 

(2015) report a goodness-of-fit R-squared of 0.19; using a similar specification, Dadgar and Trimble (2015) report 
0.59-0.72. These are different test statistics of course, but there is little guidance on what values are reasonable. 

26 For example, Minaya and Scott-Clayton (2017, Appendix Table A2) find that when including zeroes for earnings, 
OLS results in Ohio are $2,313; when excluding them, OLS results are $1,988. 

27 

Missing earnings data may generate bias for program-level analyses: college programs in which the occupational 

intent is self-employment will show relatively low returns if missing earnings are counted as zero earnings. 

28 

The U.S. Department of Education selected tax return data from W-2s (not 1099s) and only for persons who 
remained within the state in which they attended college. This selection effect reduced the tax sample by 38 percent. 

29 

Only one statewide administrative dataset includes UI records for individuals who migrate out of state: for 
Virginia, results separated by migration status are not available, although the pooled results are similar to those from 
other states that do not include out-of-state workers (Belfield & Bailey, 2017, Table 2). 
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may be a positive correlation between out-migration and degree attainment: the degree serves as 
stronger signal of productivity to a wider labor market. The returns to degrees may therefore be 
biased downward if out-of-state employment is missing. In direct analysis of migration impacts 
using NLSY97, Scott-Clayton and Wen (2017) find contrary results for associate degree holders. 
Associate degree completion does not appear to influence out-migration, but out-of-state 
migrants with associate degrees earn less than persons who stay in-state. Yet, the negative effect 
of out-migration is modest: it reduces earnings gains from an associate degree by approximately 
6 percent. 30 

More generally, locational decisions may matter. As earnings are higher in urban areas, if 
degree holders are more likely to live in cities, this will bias upward the returns to college 
(Black, Kolesnikova, & Taylor, 2009; Doyle & Skinner, 2016; Moretti, 2013). Moretti (2013) 
estimates that the four-year college wage premium is approximately one-fifth lower after 
adjusting for differences in cost-of-living between graduates and high school completers. This 
locational effect may also apply to community college students. 

Another consideration is whether it is appropriate to model earnings gains separately 
from returns net of college costs. Earnings gains and college costs may co-vary; identifying 
earnings gains alone may therefore provide an incomplete estimate of the net returns. If earnings 
gains and college costs positively co-vary, then the net returns may be very stable across student 
groups. Altonji and Zimmerman (2017) directly link evidence on earnings gains to the costs of 
college and find that more expensive programs yield higher gains such that returns overall are 
compressed. Another factor is the economic cycle: this may be an important influence on the 
returns to associate degrees (Cappelli, 2015). The difference in earnings between degree holders 
and non-completers does appear stable over the economic cycle (Belfield, 2015, Scott-Clayton & 
Wen, 2017). But there is a dip in earnings and employment probabilities for students graduating 
during a recession (Herschbein, 2012; Oreopoulos, von Wachter, & Heisz, 2012.) This dip 
reduces the returns to college net of tuition and fees (but only assuming that students enrolled in 
courses with the unchanged tuition and fees). 


6. Conclusions 

There is considerable concern that college is no longer a good investment. It is therefore 
important that estimates of the earnings gains from college are robust. The above analysis shows 
that fixed effects models are robust to most sampling restrictions and most functional form 
specifications. But there are some exceptions. Results vary depending on how missing wages are 
modeled and on how educational attainment of transfer students is addressed. Results also vary 
depending on how time trends are modeled. Nevertheless, the conclusion that there are positive 


30 


Calculated from Scott-Clayton and Wen (2017, Tables 1 and 5) based on an out-migration rate of 30 percent. 
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returns to associate degrees over non-completion remains valid. Indeed, alternative plausible 
specifications indicate that the returns are even higher than conventionally estimated. 

Fixed effects models do offer advantages over OLS estimation. They directly address 
biases due to individual-level omitted variables — even as this does not appear to be especially 
influential. As well, fixed effects models allow for new interpretations of time trends in earnings, 
including the effects of working while enrolled. Of course, these new interpretations rely on 
significantly more data than is available in cross-sectional datasets. In fact, the above studies 
suggest that the ideal window of data post-college needs to be very wide, covering at least five 
years after a student has exited college. OLS estimation of earnings premia requires relatively 
little data and, over several decades of application, has proved very robust. Alternative methods 
(such as quasi-experiments and propensity score matching) have typically found results that 
correspond to those from OLS estimation, and this also appears to be the case for fixed effects 
models (Barrow & Malamud, 2015; Carneiro et ah, 2011). There are advantages from having 
large-scale longitudinal data, but these advantages are modest. Overall, fixed effects models 
yield similar results to OLS estimates. 

Finally, even as fixed effects models appear to offer only a modest advantage in reducing 
bias and increasing precision, they do have a distinct advantage: they allow for analysis of 
earnings profiles over the period from before to after college. Given the large fluctuations in 
earnings over this period, this advantage may be significant in yielding evidence on the full 
returns to college. Students need to know that the end result is strongly positive (high earnings 
gains x years after leaving college with a degree) and that the journey to get there (leaving the 
workforce and progressing through college) is not economically burdensome. Fixed effects 
models shed new light on this journey. 
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Appendix 


Table A1 : Robustness Checks on Returns to Associate Degrees 



Washington 

Michigan 

Kentucky 





(Percent gains) 

(Dollar gains) 

(Dollar gains) 

California 

Colorado 

Ohio 


Female 

Male 

Female 

Male 

Female 

Male 

Pooled 

Pooled 

Pooled 

Baseline FE including time trend 

Sample selection 

6.5% 

2.1% 

$2,363 

$1,484 

$2,346 

$1,441 

$1,650 

$2,200 

$2,439 

Age 25+ on entry; worked pre-entry 

Age 20-50 

2.0% 

-3.0% 





$2,195 


$2,510 

Excluding enrolled in college five years later 

5.9% 

2.7% 



$2,850 

$1,850 

$2,899 

$2,226 


Including missing wages pre-entry 

Including younger persons 

Excluding younger persons 

6.9% 

2.4% 

$2,230 

$1,562 

$3,150 

$1,400 




Excluding zero credits in college 



$2,337 

$1,276 



$1,565 



Only employed 



$2,667 

$1,660 






Short window (years post-college) 



$2,346 

$1,367 



$1,196 


$1,822 

Including strong labor market attachment 



$2,210 

$1,381 




$2,449 


Including weak labor market attachment 
Adjusted for enrollment of non-completers 





$2,350 

$1,650 


$2,151 


Specifications 










Baseline excluding time trends/interactions 

8.9% 

7.2% 

$2,440 

$1,325 



$1,643 

$2,307 


Difference estimator 



$1886 

$1506 




$1,439 


Adjusting for Ashenfelter Dip (1) 

6.7% 

2.4% 



$2,350 

$1,400 

$1,662 

$2,074 


Adjusting for Ashenfelter Dip (2) 

6.7% 

2.8% 

$2396 

$1,554 

$2,350 

$1,350 





Sources: Dadgar and Trimble (2015) (Washington); Bahr et al. (2015) (Michigan); Jepsen et al. (2014) (Kentucky); Bahr (2016) (California); Turner (2016) 
(Colorado); Minaya & Scott-Clayton (2017) (Ohio). 
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Figure A1 : Percentage Decline in Earnings From Two Quarters Prior to College Entry to Quarter 
of College Entry (Kentucky, Ohio) 
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